Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 244 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.9527. Min Act: -0.0000

Data Index: 825518 (The Pile)

Max Activating Token Index: 412

Text #1

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.9102. Min Act: -0.0000

Data Index: 635424 (The Pile)

Max Activating Token Index: 431

Text #2

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.8426. Min Act: -0.0000

Data Index: 341471 (The Pile)

Max Activating Token Index: 780

Text #3

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.7762. Min Act: -0.0000

Data Index: 1376611 (The Pile)

Max Activating Token Index: 512

Text #4

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.4090. Min Act: -0.0000

Data Index: 311159 (The Pile)

Max Activating Token Index: 746

Text #5

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.4371. Min Act: -0.0000

Data Index: 1281318 (The Pile)

Max Activating Token Index: 514

Text #6

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.6091. Min Act: -0.0000

Data Index: 1675841 (The Pile)

Max Activating Token Index: 846

Text #7

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.4654. Min Act: -0.0000

Data Index: 1611728 (The Pile)

Max Activating Token Index: 1001

Text #8

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.5049. Min Act: -0.0000

Data Index: 237441 (The Pile)

Max Activating Token Index: 710

Text #9

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.4069. Min Act: -0.0000

Data Index: 1661736 (The Pile)

Max Activating Token Index: 926

Text #10

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.2508. Min Act: -0.0000

Data Index: 1515757 (The Pile)

Max Activating Token Index: 466

Text #11

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.3231. Min Act: -0.0000

Data Index: 124613 (The Pile)

Max Activating Token Index: 638

Text #12

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.4660. Min Act: -0.0000

Data Index: 1494336 (The Pile)

Max Activating Token Index: 331

Text #13

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.2370. Min Act: -0.0000

Data Index: 1459179 (The Pile)

Max Activating Token Index: 288

Text #14

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.3026. Min Act: -0.0000

Data Index: 953985 (The Pile)

Max Activating Token Index: 340

Text #15

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.2794. Min Act: -0.0000

Data Index: 241257 (The Pile)

Max Activating Token Index: 913

Text #16

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.2947. Min Act: -0.0000

Data Index: 864700 (The Pile)

Max Activating Token Index: 1002

Text #17

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.2468. Min Act: -0.0000

Data Index: 941532 (The Pile)

Max Activating Token Index: 592

Text #18

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.0932. Min Act: -0.0000

Data Index: 206255 (The Pile)

Max Activating Token Index: 266

Text #19

Max Range: 1.9527. Min Range: -1.9527

Max Act: 1.2562. Min Act: -0.0000

Data Index: 279220 (The Pile)

Max Activating Token Index: 262