Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4716 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.9938. Min Act: -0.0000

Data Index: 1454804 (The Pile)

Max Activating Token Index: 494

Text #1

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.8295. Min Act: -0.0000

Data Index: 1463560 (The Pile)

Max Activating Token Index: 339

Text #2

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.6156. Min Act: -0.0000

Data Index: 1615619 (The Pile)

Max Activating Token Index: 888

Text #3

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.5532. Min Act: -0.0000

Data Index: 1436891 (The Pile)

Max Activating Token Index: 127

Text #4

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.6167. Min Act: -0.0000

Data Index: 1096335 (The Pile)

Max Activating Token Index: 876

Text #5

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.6098. Min Act: -0.0000

Data Index: 543963 (The Pile)

Max Activating Token Index: 912

Text #6

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.3561. Min Act: -0.0000

Data Index: 1832010 (The Pile)

Max Activating Token Index: 437

Text #7

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.2663. Min Act: -0.0000

Data Index: 644443 (The Pile)

Max Activating Token Index: 771

Text #8

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.4064. Min Act: -0.0000

Data Index: 1722621 (The Pile)

Max Activating Token Index: 752

Text #9

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.4406. Min Act: -0.0000

Data Index: 919259 (The Pile)

Max Activating Token Index: 473

Text #10

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.1770. Min Act: -0.0000

Data Index: 615888 (The Pile)

Max Activating Token Index: 755

Text #11

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.3978. Min Act: -0.0000

Data Index: 96105 (The Pile)

Max Activating Token Index: 693

Text #12

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.1888. Min Act: -0.0000

Data Index: 1166435 (The Pile)

Max Activating Token Index: 512

Text #13

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.2881. Min Act: -0.0000

Data Index: 1314486 (The Pile)

Max Activating Token Index: 104

Text #14

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.3642. Min Act: -0.0000

Data Index: 1887891 (The Pile)

Max Activating Token Index: 468

Text #15

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.3269. Min Act: -0.0000

Data Index: 1641675 (The Pile)

Max Activating Token Index: 451

Text #16

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.2595. Min Act: -0.0000

Data Index: 1311835 (The Pile)

Max Activating Token Index: 418

Text #17

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.2822. Min Act: -0.0000

Data Index: 1980463 (The Pile)

Max Activating Token Index: 439

Text #18

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.0503. Min Act: -0.0000

Data Index: 1388681 (The Pile)

Max Activating Token Index: 864

Text #19

Max Range: 1.9938. Min Range: -1.9938

Max Act: 1.3413. Min Act: -0.0000

Data Index: 173934 (The Pile)

Max Activating Token Index: 546