Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3837 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.8401. Min Act: -0.0000

Data Index: 718206 (The Pile)

Max Activating Token Index: 698

Text #1

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.8090. Min Act: -0.0000

Data Index: 1581853 (The Pile)

Max Activating Token Index: 630

Text #2

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.5191. Min Act: -0.0000

Data Index: 1950814 (The Pile)

Max Activating Token Index: 684

Text #3

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.6536. Min Act: -0.0000

Data Index: 262555 (The Pile)

Max Activating Token Index: 479

Text #4

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.4321. Min Act: -0.0000

Data Index: 977965 (The Pile)

Max Activating Token Index: 402

Text #5

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.4400. Min Act: -0.0000

Data Index: 1376338 (The Pile)

Max Activating Token Index: 387

Text #6

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.4273. Min Act: -0.0000

Data Index: 1134761 (The Pile)

Max Activating Token Index: 478

Text #7

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.1032. Min Act: -0.0000

Data Index: 1141957 (The Pile)

Max Activating Token Index: 422

Text #8

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.2435. Min Act: -0.0000

Data Index: 1718590 (The Pile)

Max Activating Token Index: 918

Text #9

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.3357. Min Act: -0.0000

Data Index: 1710741 (The Pile)

Max Activating Token Index: 885

Text #10

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.3374. Min Act: -0.0000

Data Index: 853546 (The Pile)

Max Activating Token Index: 492

Text #11

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.4799. Min Act: -0.0000

Data Index: 927634 (The Pile)

Max Activating Token Index: 682

Text #12

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.1672. Min Act: -0.0000

Data Index: 757331 (The Pile)

Max Activating Token Index: 870

Text #13

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.2906. Min Act: -0.0000

Data Index: 331826 (The Pile)

Max Activating Token Index: 449

Text #14

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.2923. Min Act: -0.0000

Data Index: 112560 (The Pile)

Max Activating Token Index: 314

Text #15

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.2705. Min Act: -0.0000

Data Index: 59360 (The Pile)

Max Activating Token Index: 674

Text #16

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.1906. Min Act: -0.0000

Data Index: 1390549 (The Pile)

Max Activating Token Index: 793

Text #17

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.1959. Min Act: -0.0000

Data Index: 809914 (The Pile)

Max Activating Token Index: 623

Text #18

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.1662. Min Act: -0.0000

Data Index: 537527 (The Pile)

Max Activating Token Index: 806

Text #19

Max Range: 3.8401. Min Range: -3.8401

Max Act: 3.2845. Min Act: -0.0000

Data Index: 339959 (The Pile)

Max Activating Token Index: 307