Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2614 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.2528. Min Act: -0.0000

Data Index: 67938 (The Pile)

Max Activating Token Index: 126

Text #1

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.2149. Min Act: -0.0000

Data Index: 1153994 (The Pile)

Max Activating Token Index: 317

Text #2

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.2167. Min Act: -0.0000

Data Index: 144589 (The Pile)

Max Activating Token Index: 722

Text #3

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.2446. Min Act: -0.0000

Data Index: 839039 (The Pile)

Max Activating Token Index: 969

Text #4

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.1868. Min Act: -0.0000

Data Index: 745256 (The Pile)

Max Activating Token Index: 807

Text #5

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.1854. Min Act: -0.0000

Data Index: 1566976 (The Pile)

Max Activating Token Index: 407

Text #6

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.2004. Min Act: -0.0000

Data Index: 1408337 (The Pile)

Max Activating Token Index: 757

Text #7

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.1782. Min Act: -0.0000

Data Index: 44044 (The Pile)

Max Activating Token Index: 797

Text #8

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.2000. Min Act: -0.0000

Data Index: 488689 (The Pile)

Max Activating Token Index: 663

Text #9

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.1973. Min Act: -0.0000

Data Index: 1644674 (The Pile)

Max Activating Token Index: 667

Text #10

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.1821. Min Act: -0.0000

Data Index: 963888 (The Pile)

Max Activating Token Index: 457

Text #11

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.1672. Min Act: -0.0000

Data Index: 1832578 (The Pile)

Max Activating Token Index: 492

Text #12

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.1776. Min Act: -0.0000

Data Index: 1958447 (The Pile)

Max Activating Token Index: 824

Text #13

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.1825. Min Act: -0.0000

Data Index: 656383 (The Pile)

Max Activating Token Index: 851

Text #14

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.1646. Min Act: -0.0000

Data Index: 1582293 (The Pile)

Max Activating Token Index: 596

Text #15

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.1610. Min Act: -0.0000

Data Index: 21307 (The Pile)

Max Activating Token Index: 183

Text #16

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.1666. Min Act: -0.0000

Data Index: 1089448 (The Pile)

Max Activating Token Index: 628

Text #17

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.1641. Min Act: -0.0000

Data Index: 554325 (The Pile)

Max Activating Token Index: 832

Text #18

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.1748. Min Act: -0.0000

Data Index: 603841 (The Pile)

Max Activating Token Index: 853

Text #19

Max Range: 0.2528. Min Range: -0.2528

Max Act: 0.1441. Min Act: -0.0000

Data Index: 1291763 (The Pile)

Max Activating Token Index: 126