Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3471 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.4354. Min Act: -0.0000

Data Index: 748983 (The Pile)

Max Activating Token Index: 161

Text #1

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.3962. Min Act: -0.0000

Data Index: 265908 (The Pile)

Max Activating Token Index: 484

Text #2

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.3747. Min Act: -0.0000

Data Index: 1604533 (The Pile)

Max Activating Token Index: 711

Text #3

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.4188. Min Act: -0.0000

Data Index: 727461 (The Pile)

Max Activating Token Index: 420

Text #4

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.3805. Min Act: -0.0000

Data Index: 1129877 (The Pile)

Max Activating Token Index: 787

Text #5

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.3543. Min Act: -0.0000

Data Index: 312053 (The Pile)

Max Activating Token Index: 898

Text #6

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.3845. Min Act: -0.0000

Data Index: 1400837 (The Pile)

Max Activating Token Index: 536

Text #7

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.3616. Min Act: -0.0000

Data Index: 142644 (The Pile)

Max Activating Token Index: 592

Text #8

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.3544. Min Act: -0.0000

Data Index: 1303477 (The Pile)

Max Activating Token Index: 51

Text #9

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.3571. Min Act: -0.0000

Data Index: 1791018 (The Pile)

Max Activating Token Index: 572

Text #10

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.3343. Min Act: -0.0000

Data Index: 1312330 (The Pile)

Max Activating Token Index: 745

Text #11

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.3527. Min Act: -0.0000

Data Index: 1537081 (The Pile)

Max Activating Token Index: 111

Text #12

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.3369. Min Act: -0.0000

Data Index: 445002 (The Pile)

Max Activating Token Index: 325

Text #13

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.3146. Min Act: -0.0000

Data Index: 1280449 (The Pile)

Max Activating Token Index: 387

Text #14

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.3702. Min Act: -0.0000

Data Index: 1158204 (The Pile)

Max Activating Token Index: 426

Text #15

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.3189. Min Act: -0.0000

Data Index: 1094989 (The Pile)

Max Activating Token Index: 849

Text #16

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.2998. Min Act: -0.0000

Data Index: 527534 (The Pile)

Max Activating Token Index: 493

Text #17

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.3290. Min Act: -0.0000

Data Index: 480162 (The Pile)

Max Activating Token Index: 355

Text #18

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.3079. Min Act: -0.0000

Data Index: 1739446 (The Pile)

Max Activating Token Index: 490

Text #19

Max Range: 0.4354. Min Range: -0.4354

Max Act: 0.3187. Min Act: -0.0000

Data Index: 1614641 (The Pile)

Max Activating Token Index: 802