Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 2261 in Layer 10

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.9900. Min Act: -0.0000

Data Index: 214886 (C4 (Web Text))

Max Activating Token Index: 979

Text #1

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.8339. Min Act: -0.0000

Data Index: 1252866 (C4 (Web Text))

Max Activating Token Index: 283

Text #2

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.6865. Min Act: -0.0000

Data Index: 823614 (C4 (Web Text))

Max Activating Token Index: 139

Text #3

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.7258. Min Act: -0.0000

Data Index: 269609 (C4 (Web Text))

Max Activating Token Index: 421

Text #4

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.6774. Min Act: -0.0000

Data Index: 350546 (C4 (Web Text))

Max Activating Token Index: 178

Text #5

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.6042. Min Act: -0.0000

Data Index: 358477 (C4 (Web Text))

Max Activating Token Index: 359

Text #6

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.6179. Min Act: -0.0000

Data Index: 854392 (C4 (Web Text))

Max Activating Token Index: 581

Text #7

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.5540. Min Act: -0.0000

Data Index: 38062 (C4 (Web Text))

Max Activating Token Index: 200

Text #8

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.5195. Min Act: -0.0000

Data Index: 475659 (C4 (Web Text))

Max Activating Token Index: 371

Text #9

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.4279. Min Act: -0.0000

Data Index: 488698 (C4 (Web Text))

Max Activating Token Index: 441

Text #10

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.5012. Min Act: 0.0000

Data Index: 812872 (C4 (Web Text))

Max Activating Token Index: 104

Text #11

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.4871. Min Act: -0.0000

Data Index: 280880 (C4 (Web Text))

Max Activating Token Index: 465

Text #12

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.4774. Min Act: -0.0000

Data Index: 668868 (C4 (Web Text))

Max Activating Token Index: 883

Text #13

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.4319. Min Act: -0.0000

Data Index: 1083063 (C4 (Web Text))

Max Activating Token Index: 259

Text #14

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.5122. Min Act: -0.0000

Data Index: 103103 (C4 (Web Text))

Max Activating Token Index: 893

Text #15

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.4719. Min Act: -0.0000

Data Index: 1310768 (C4 (Web Text))

Max Activating Token Index: 280

Text #16

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.3754. Min Act: -0.0000

Data Index: 1197887 (C4 (Web Text))

Max Activating Token Index: 59

Text #17

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.3809. Min Act: -0.0000

Data Index: 32334 (C4 (Web Text))

Max Activating Token Index: 158

Text #18

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.3443. Min Act: -0.0000

Data Index: 515479 (C4 (Web Text))

Max Activating Token Index: 899

Text #19

Max Range: 1.9900. Min Range: -1.9900

Max Act: 1.3497. Min Act: -0.0000

Data Index: 1254370 (C4 (Web Text))

Max Activating Token Index: 906